Display method for A-pillar-mounted display assemblies, and display device of A-pillars of vehicle, and storage medium thereof

ABSTRACT

Provided is a display method for A-pillar-mounted display assemblies of a vehicle. The method includes: acquiring facial posture information of a driver of the vehicle in a camera coordinate system by any one driver monitoring assembly of at least one driver monitoring assembly, determining a visual field of the driver based on a gaze direction of the driver and the eye position of the driver, acquiring coordinates of the two display assemblies in a world coordinate system, converting the coordinates of the two display assemblies in the world coordinate system into coordinates in the camera coordinate system based on a first conversion relationship, determining whether any one display assembly of the two display assemblies is within the visual field, and capturing an external image of the vehicle captured by the imaging assembly.

CROSS-PREFERENCE TO RELATED APPLICATIONS

This application is a national phase application based onPCT/CN2021/080063, field on Mar. 10, 2021, which claims priority of theChinese Patent Application No. 202010286777.4, filed on Apr. 13, 2020and entitled “DISPLAY METHOD FOR A-PILLARS-MOUNTED DISPLAY ASSEMBLIES,AND DISPLAY SYSTEM, AND DISPLAY DEVICE OF A-PILLARS OF VEHICLE, ANDSTORAGE MEDIUM THEREOF,” the entire contents of which are incorporatedherein by reference in their entireties.

TECHNICAL FIELD

The present disclosure relates to the field of display technologies, andin particular relates to a display method for A-pillar-mounted displayassemblies of a vehicle, a display device, and a storage medium thereof.

BACKGROUND

A-pillars of a vehicle refers to pillars between a front windshield andtwo front doors of the vehicle. The A-pillar is one of main partsproviding a structural strength for the vehicle. However, the A-pillarmay also block a visual field of a driver to a certain extent.

At present, a display method for A-pillars of a vehicle is applicable toa vehicle. The vehicle includes an imaging assembly for capturing imagesof the exterior of the vehicle, a display assembly is disposed on theA-pillar of the vehicle, and a monitoring assembly for acquiring an eyeposition of the driver is disposed in the vehicle. In this method, ascreen with a larger viewing angle may be acquired by an external cameraassembly, then whether the driver is gazing at the display assembly isdetermined based on spatial positions of the display assembly and thedriver's eyes, and the display assembly is controlled.

SUMMARY

According to a first aspect of the present disclosure, a display methodfor A-pillar-mounted display assemblies of a vehicle is provided. Thedisplay method is applicable to the vehicle, and the vehicle includes animaging assembly, at least one driver monitoring assembly, and twodisplay assemblies disposed respectively on two A-pillars at the insideof the vehicle. The display method includes:

-   -   acquiring facial posture information of a driver of the vehicle        in a camera coordinate system by any one driver monitoring        assembly of the at least one driver monitoring assembly, wherein        the facial posture information includes a gaze direction of the        driver and an eye position of the driver;    -   determining a visual field of the driver based on the gaze        direction of the driver and the eye position of the driver;    -   acquiring coordinates of the two display assemblies in a world        coordinate system;    -   converting the coordinates of the two display assemblies in the        world coordinate system into coordinates in the camera        coordinate system based on a first conversion relationship,        wherein the first conversion relationship is determined by a        second conversion relationship and a third conversion        relationship, the second conversion relationship being a        conversion relationship between the coordinates in the camera        coordinate system of the any one driver monitoring assembly and        coordinates in a target feature model in the vehicle, and the        third conversion relationship being a conversion relationship        between the coordinates in the target feature model and the        coordinates in the world coordinate system;    -   determining, based on the coordinates of the two display        assemblies in the camera coordinate system, whether any one        display assembly of the two display assemblies is within the        visual field; and    -   capturing, based on the eye position of the driver, an external        image of the vehicle captured by the imaging assembly in        response to the any one display assembly being within the visual        field, and displaying the captured image on the any one display        assembly.

Optionally, prior to acquiring the facial posture information of thedriver of the vehicle in the camera coordinate system by the any onedriver monitoring assembly of the at least one driver monitoringassembly, the method further includes:

-   -   establishing the world coordinate system;    -   acquiring a target image of the target feature model in the        vehicle by the any one driver monitoring assembly;    -   determining the second conversion relationship between the        coordinates in the target feature model and the coordinates in        the camera coordinate system based on the target image of the        target feature model;    -   acquiring the third conversion relationship between the        coordinates in the target feature model and the coordinates in        the world coordinate system; and    -   determining the first conversion relationship based on the third        conversion relationship and the second conversion relationship.

Optionally, the target feature model includes a standard facial featuremodel, wherein the standard facial feature model includes a plurality offeature points and three-dimensional coordinates of each of the featurepoints; and

-   -   acquiring the target image of the target feature model in the        vehicle by the any one driver monitoring assembly includes:    -   acquiring a target image of the standard facial feature model by        the any one driver monitoring assembly; and    -   determining the second conversion relationship between the        coordinates in the target feature model and the coordinates in        the camera coordinate system based on the target image of the        target feature model includes:    -   determining planar coordinates of the plurality of feature        points in the standard facial feature model based on the target        image;    -   acquiring an internal parameter matrix and a distortion        coefficient of the any one driver monitoring assembly by        calibrating the any one driver monitoring assembly; and    -   acquiring the second conversion relationship by solving a        perspective-n-point problem based on the internal parameter        matrix, the distortion coefficient, the three-dimensional        coordinates, and the planar coordinates.

Optionally, acquiring the third conversion relationship between thecoordinates in the target feature model and the coordinates in the worldcoordinate system includes:

-   -   acquiring a rotation matrix and a translation vector between the        standard facial feature model and the world coordinate system;        and    -   determining the third conversion relationship based on the        rotation matrix and the translation vector.

Optionally, prior to acquiring the rotation matrix and the translationvector between the standard facial feature model and the worldcoordinate system, the method further includes:

-   -   adjusting a coordinate system of the standard facial feature        model such that one coordinate axis in the coordinate system of        the standard facial feature model is parallel to one coordinate        axis in the world coordinate system.

Optionally, the vehicle includes two driver monitoring assembliesrespectively disposed on two A-pillars at the inside of the vehicle,wherein the two driver monitoring assemblies include the any one drivermonitoring assembly and the other driver monitoring assembly, andacquiring the facial posture information of the driver of the vehicle inthe camera coordinate system by the any one driver monitoring assemblyof the at least one driver monitoring assembly includes:

-   -   activating the two driver monitoring assemblies alternately; and    -   continuously acquiring the facial posture information by the any        one driver monitoring assembly of the two driver monitoring        assemblies, and deactivating the other driver monitoring        assembly of the two driver monitoring assemblies other than the        any one driver monitoring assembly, in response to the any one        driver monitoring assembly of the two driver monitoring        assemblies detecting the facial posture information.

Optionally, the target feature model includes a standard facial featuremodel, wherein the standard facial feature model includes a plurality offeature points and three-dimensional coordinates of each of the featurepoints; and

-   -   prior to acquiring the facial posture information of the driver        of the vehicle in the camera coordinate system by the any one        driver monitoring assembly of the at least one driver monitoring        assembly, the method further includes:    -   establishing the world coordinate system;    -   acquiring a target image of the standard facial feature model by        the any one driver monitoring assembly;    -   determining planar coordinates of the plurality of feature        points in the standard facial feature model based on the target        image;    -   acquiring an internal parameter matrix and a distortion        coefficient of the any one driver monitoring assembly by        calibrating the any one driver monitoring assembly;    -   acquiring the second conversion relationship by solving a        perspective-n-point problem based on the internal parameter        matrix, the distortion coefficient, the three-dimensional        coordinates, and the planar coordinates;    -   acquiring a rotation matrix and a translation vector between the        standard facial feature model and the world coordinate system;    -   determining the third conversion relationship based on the        rotation matrix and the translation vector; and    -   determining the first conversion relationship based on the third        conversion relationship and the second conversion relationship.

Optionally, the imaging assembly includes imaging sub-assembliesdisposed on the outside of the two A-pillars, wherein a line between theimaging sub-assembly on any one A-pillar of the two A-pillars and thedriver's eyes is intersected with the display assembly on the any oneA-pillar.

According to another aspect of the present disclosure, a display systemfor A-pillars of a vehicle is provided. The vehicle includes an imagingassembly, at least one driver monitoring assembly, and two displayassemblies disposed respectively on two A-pillars at the inside of thevehicle. The display system includes:

-   -   a posture acquiring module, configured to acquire facial posture        information of a driver of the vehicle in a camera coordinate        system by any one driver monitoring assembly of the at least one        driver monitoring assembly, wherein the facial posture        information including a gaze direction of the driver and an eye        position of the driver;    -   a visual field determining module, configured to determine a        visual field of the driver based on the gaze direction of the        driver and the eye position of the driver;    -   a coordinate acquiring module, configured to acquire coordinates        of the two display assemblies in a world coordinate system;    -   a coordinate converting module, configured to convert the        coordinates of the two display assemblies in the world        coordinate system into coordinates in the camera coordinate        system based on a first conversion relationship, wherein the        first conversion relationship is determined by a second        conversion relationship and a third conversion relationship, the        second conversion relationship being a conversion relationship        between the coordinates in the camera coordinate system of the        any one driver monitoring assembly and coordinates in a target        feature model in the vehicle, and the third conversion        relationship being a conversion relationship between the        coordinates in the target feature model and the coordinates in        the world coordinate system;    -   a visual field determining module, configured to determine,        based on the coordinates of the two display assemblies in the        camera coordinate system, whether any one display assembly of        the two display assemblies is within the visual field; and    -   an image displaying module, configured to capture, based on the        eye position of the driver, an external image of the vehicle        captured by the imaging assembly in response to the any one        display assembly being within the visual field, and display the        captured image on the any one display assembly.

According to still another aspect of the present disclosure, a displaydevice for A-pillars of a vehicle is provided. The display deviceincludes a processor and a memory storing at least one instruction, atleast one program, a code set or an instruction set therein; wherein theprocessor, when loading and executing the at least one instruction, theat least one program, the code set or the instruction set, is caused toperform the above-mentioned display methods for the A-pillar-mounteddisplay assemblies of the vehicle.

According to still another aspect of the present disclosure, anon-volatile computer storage medium is provided. The non-volatilecomputer storage medium stores at least one instruction, at least oneprogram, a code set or an instruction set; wherein the at least oneinstruction, the at least one program, the code set or the instructionset, when loaded and executed by a processor, causes the processor toperform the above-mentioned display methods for the A-pillar-mounteddisplay assemblies of the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

For clearer descriptions of the technical solutions in the embodimentsof the present disclosure, the following briefly introduces theaccompanying drawings required for describing the embodiments.Apparently, the accompanying drawings in the following description showmerely some embodiments of the present disclosure, and a person ofordinary skill in the art may still derive other drawings from theseaccompanying drawings without creative efforts.

FIG. 1 is a schematic diagram of an internal structure of a vehicleaccording to an embodiment of the present disclosure;

FIG. 2 is a left view of the A-pillar of the vehicle shown in FIG. 1 ;

FIG. 3 is a flowchart of a display method for A-pillar-mounted displayassemblies of a vehicle according to an embodiment of the presentdisclosure;

FIG. 4 is a flowchart of another display method for A-pillar-mounteddisplay assemblies of a vehicle according to an embodiment of thepresent disclosure;

FIG. 5 is a schematic diagram of a standard facial feature model in theembodiment shown in FIG. 4 ;

FIG. 6 is a flowchart of determining a second conversion relationship inthe embodiment shown in FIG. 4 ;

FIG. 7 is a flowchart of determining a third conversion relationship inthe embodiment shown in FIG. 4 ;

FIG. 8 is a schematic diagram of conversion relationships betweenvarious coordinate systems in the embodiment shown in FIG. 4 ;

FIG. 9 is a schematic diagram of a visual field of a driver in theembodiment shown in FIG. 4 ;

FIG. 10 is a schematic diagram in the case that the display assembly onthe left A-pillar of the vehicle in the embodiment shown in FIG. 4 iswithin a visual field of a driver; and

FIG. 11 is a structural block diagram of a display system for A-pillarsof a vehicle according to an embodiment of the present disclosure.

The specific embodiments of the present disclosure are illustratedexplicitly through the above drawings, and are described in detaillater. These drawings and text descriptions are not intended to limitthe scope of the concept of the present disclosure in any way, but toexplain the concept of the present disclosure to those skilled in theart by referring to specific embodiments.

DETAILED DESCRIPTION

In the related art, it is relatively difficult to determine whether thedriver is gazing at the display assembly directly based on the spatialpositions of the display assembly and the driver's eyes.

Embodiments of the present disclosure are described in further detailwith reference to the accompanying drawings, for clear description ofthe objectives, technical solutions, and advantages of the presentdisclosure.

FIG. 1 is a schematic diagram of an internal structure of a vehicleaccording to an embodiment of the present disclosure. The vehicleincludes at least one driver monitoring assembly 11 and two displayassemblies 13 respectively disposed on two A-pillars 12 at the inside ofthe vehicle.

The driver monitoring assembly 11 may acquire images of the interior ofthe vehicle. Optionally, the vehicle includes two driver monitoringassemblies 11, and the two driver monitoring assemblies 11 arerespectively disposed on two A-pillars at the inside of the vehicle.Each monitoring assembly 11 may include a camera.

The two display assemblies 13 may be respectively disposed on the twoA-pillars. Optionally, each display assembly may include a flexibledisplay panel. Because a side of the A-pillar facing the interior of thevehicle is usually not a regular plane, the display assembly formed bythe flexible display panel may be disposed on the A-pillar moreconveniently. For example, the flexible display panel may be an organiclight-emitting diode (OLED) display panel.

In embodiments of the present disclosure, the inside of the vehicle mayrefer to the side facing the inner space of the vehicle.

Because the driver monitoring assembly needs to monitor a driver's face,the driver may appear in an image range of the driver monitoringassembly. However, the display assembly is disposed on the A-pillar. Inorder to better acquire the driver's facial information, it is difficultfor the display assembly and the driver to appear in an image of thedriver monitoring assembly simultaneously. A subsequent problem is thatit is difficult to directly acquire coordinates of the display assemblyin a camera coordinate system of the driver monitoring assembly, and itis also difficult to determine whether the display assembly is withinthe visual field of the driver.

In addition, the vehicle according to the embodiment of the presentdisclosure may further include some other parts, such as an engine, agearbox, a chassis, and a vehicle housing, which are not repeated in theembodiment of the present disclosure.

The embodiment of the present disclosure provides a method in which atarget feature model may be placed in the vehicle. The target featuremodel is disposed within the image range of the driver monitoringassembly, and a conversion relationship between the camera coordinatesystem of the driver monitoring assembly and a world coordinate systemis acquired by the target feature model as a transition. In this way,world coordinates of the display assembly (the world coordinates may bedirectly acquired by measurement) may be converted into cameracoordinates, so as to easily determine whether the driver is gazing at adisplay assembly.

As shown in FIG. 2 , FIG. 2 is a left view of the A-pillar 12 of thevehicle shown in FIG. 1 . The vehicle further includes an imagingassembly 14 disposed on the outside of the A-pillar. The imagingassembly 14 may include a camera for acquiring images of the exterior ofthe vehicle. Optionally, the imaging assembly 14 includes imagingsub-assemblies 141 disposed on the outside of two A-pillars (FIG. 2shows the imaging sub-assembly on one of the A-pillars at the outside ofthe vehicle). In the embodiments of the present disclosure, the outsideof the A-pillar may refer to a side of the A-pillar away from theinterior of the vehicle.

Alternatively, a line between the imaging sub-assembly 141 on eitherA-pillar of the two A-pillars and the driver's eyes E is intersectedwith the display assembly 13 on the either A-pillar 12. By such astructure, the imaging sub-assembly and the display assembly aredisposed in the same orientation of the driver, and an external imagecaptured by the imaging sub-assembly may better conform to an externalimage blocked by the A-pillar.

FIG. 3 is a flowchart of a display method for A-pillar-mounted displayassemblies of a vehicle according to an embodiment of the presentdisclosure. The method may be applicable to the vehicle shown in FIG. 1. The method may include following steps.

In 301, facial posture information of the driver of the vehicle in thecamera coordinate system is acquired by any one driver monitoringassembly of the at least one driver monitoring assembly, wherein thefacial posture information includes a gaze direction of the driver andan eye position of the driver.

In 302, a visual field of the driver is determined based on the gazedirection of the driver and the eye position of the driver.

In 303, coordinates of the two display assemblies in the worldcoordinate system is acquired.

In 304, the coordinates of the two display assemblies in the worldcoordinate system are converted into coordinates in the cameracoordinate system based on a first conversion relationship, wherein thefirst conversion relationship is determined by a second conversionrelationship and a third conversion relationship, the second conversionrelationship is a conversion relationship between the coordinates in thecamera coordinate system of the any one driver monitoring assembly andcoordinates in a target feature model in the vehicle, and the thirdconversion relationship is a conversion relationship between thecoordinates in the target feature model and the coordinates in the worldcoordinate system.

In 305, whether any one display assembly of the two display assembliesis within the visual field is determined based on the coordinates of thetwo display assemblies in the camera coordinate system.

In 306, an external image of the vehicle captured by the imagingassembly is captured based on the eye position of the driver in responseto the any one display assembly being within the visual field, and thecaptured image is displayed by the any one display assembly.

In summary, in the display method for the A-pillar-mounted displayassemblies of the vehicle according to embodiments of the presentdisclosure, by a conversion relationship between a coordinate system ofa target feature model and a camera coordinate system of a drivermonitoring assembly and a conversion relationship between the targetfeature model and a world coordinate system, a conversion relationshipbetween the camera coordinate system and the world coordinate system isacquired, the coordinates of the display assembly in the worldcoordinate system can be converted into the coordinates in the cameracoordinate system, such that whether a driver is gazing at the displayassembly can be simply determined. In this way, the problem in therelated art that it is difficult to determine, directly based on spatialpositions of the driver's eyes and the display assembly, whether thedriver is gazing at the display assembly is solved, and the effect ofreducing the difficulty of determining whether the driver is gazing atthe display assembly is achieved.

FIG. 4 is a flowchart of another display method for A-pillar-mounteddisplay assemblies of a vehicle according to an embodiment of thepresent disclosure. The method may be applicable to the vehicle shown inFIG. 1 . The method may include following steps.

In 401, a world coordinate system is established.

The world coordinate system may be an absolute coordinate system, andthis coordinate system may include an origin and a plurality of axespassing through the origin. All objects in real space may be measured toacquire their coordinates in the world coordinate system.

In 402, a target image of the target feature model in the vehicle iscaptured by any one driver monitoring assembly.

The target feature model may include a plurality of feature points, andmay have a target feature coordinate system, and the coordinates of theplurality of feature points in the target feature coordinate system mayall be known (for example, acquired by measurement). These coordinatesmay be three-dimensional coordinates of the feature points in the targetfeature model.

Optionally, the target feature model may include a standard facialfeature model, and the standard facial feature model may be a commonmodel in the art. As shown in FIG. 5 , FIG. 5 is a schematic diagram ofa standard facial feature model. The model may include 68 featurepoints, where each small circle is a feature point, andthree-dimensional coordinates of the 68 feature points in the coordinatesystem of the standard facial feature model are known.

In the case that the target feature model includes the standard facialfeature model, the target image of the standard facial feature model maybe captured by any one driver monitoring assembly.

In 403, the second conversion relationship between the coordinates inthe target feature model and the coordinates in the camera coordinatesystem of the driver monitoring assembly is determined based on thetarget image of the target feature model.

The driver monitoring assembly may include a camera, and thus may have acamera coordinate system. For example, the camera coordinate system maybe a three-dimensional rectangular coordinate system established with afocus center of the camera as the origin and an optical axis of thecamera as the z axis.

In the case that the target feature model includes the standard facialfeature model, as shown in FIG. 6 , step 403 may include following threesteps.

In 4031, planar coordinates of the plurality of feature points in thestandard facial feature model are determined based on the target image.

Because the target image is a plane image, the planar coordinates of theplurality of feature points in the standard facial feature model may bedetermined based on the target image.

In 4032, an internal parameter matrix and a distortion coefficient ofthe any one driver monitoring assembly are acquired by calibrating theany one driver monitoring assembly.

In the embodiment of the present disclosure, a calibration method forthe driver monitoring assembly may be a traditional camera calibrationmethod, an active vision camera calibration method, or a cameraself-calibration method, etc., which is not limited in the embodiment ofthe present disclosure.

In 4033, the second conversion relationship is acquired by solving aperspective-n-point problem based on the internal parameter matrix, thedistortion coefficient, the three-dimensional coordinates, and theplanar coordinates.

A rotation matrix R1 and a translation vector t1 may be acquired bysolving the perspective-n-point problem based on the internal parametermatrix, the distortion coefficient, the three-dimensional coordinates,and the planar coordinates. The second conversion relationship betweenthe coordinates in the target feature model and the coordinates in thecamera coordinate system of the driver monitoring assembly may beacquired based on the rotation matrix R1 and translation vector t1. Forexample, the second conversion relationship may be acquired by a firstconversion formula as follows:P _(c) =R1×P _(f) +t1

Pc represents coordinates in the camera coordinate system, and P_(f)represents coordinates in the target feature model.

For details of solving the perspective-n-point (PnP) problem, referencemay be made to related technologies, which are not limited in theembodiment of the present disclosure.

In 404, the third conversion relationship between the coordinates in thetarget feature model and the coordinates in the world coordinate systemis acquired.

In the case that the target feature model includes the standard facialfeature model, as shown in FIG. 7 , step 404 may include following threesteps.

In 4041, a coordinate system of the standard facial feature model isadjusted such that one coordinate axis in the coordinate system of thestandard facial feature model is parallel to one coordinate axis in theworld coordinate system.

In the case that one coordinate axis in the coordinate system of thestandard facial feature model is parallel to one coordinate axis in theworld coordinate system, a subsequent calculation amount for therotation matrix may be reduced. For example, a frontal axis of thestandard facial feature model may be parallel to an x axis, a y axis, ora z axis in the world coordinate system.

In 4042, a rotation matrix and a translation vector between coordinatesof the standard facial feature model and the world coordinate system areacquired.

The rotation matrix R2 may be calculated based on a rotationrelationship between the axes in the coordinate system of the standardfacial feature model and the axes in the world coordinate system, andthe translation vector t2 may be acquired by measurement.

In 4043, the third conversion relationship is determined based on therotation matrix and the translation vector.

The third conversion relationship may be acquired based on a secondconversion formula as follows:P _(f) =R2×P _(w) +t2

Pw represents coordinates in the world coordinate system.

In 405, the first conversion relationship is determined based on thethird conversion relationship and the second conversion relationship.

In the case that the third conversion relationship and the secondconversion relationship are required, the first conversion relationshipbetween the coordinates in the camera coordinate system of the any onedriver monitoring assembly and the coordinates in the world coordinatesystem may be acquired based on these two conversion relationships.

FIG. 8 is a schematic diagram of conversion relationships betweenvarious coordinate systems according to the embodiments of the presentdisclosure. As shown in FIG. 8 , the coordinates of the world coordinatesystem are Pw, and the axes of the world coordinate system include the xaxis, the y axis and the z axis, and the conversion relationship betweenPw and coordinates Pf in the standard facial model coordinate system maybe acquired in response to acquiring the rotation matrix R2 and thetranslation vector t2. The axes in the standard facial model coordinatesystem include an x′ axis, a y′ axis and a z′ axis, and the conversionrelationship between Pf and coordinates Pc in the camera coordinatesystem may be acquired in response to acquiring the rotation matrix R1and the translation vector t1. The axes in the camera coordinate systeminclude an x″ axis, a y″ axis, and a z″ axis.

Steps 402 to 405 are the process of acquiring the first conversionrelationship between the coordinates in the camera coordinate system ofany one driver monitoring assembly and the coordinates in the worldcoordinate system, and the first conversion relationship between thecoordinates in the camera coordinate system of each driver monitoringassembly in the vehicle and the coordinates in the world coordinatesystem may be acquired by the method according to step 402 to step 405.

In 406, the coordinates of the two display assemblies in the worldcoordinate system are acquired.

The coordinates of the two display assemblies in the world coordinatesystem may be acquired by measurement.

In 407, the coordinates of the two display assemblies in the worldcoordinate system are converted to the coordinates in the cameracoordinate system based on the first conversion relationship.

The coordinates of the two display assemblies in the world coordinatesystem may be converted to coordinates in the camera coordinate systemby the first conversion relationship acquired in step 405.

Steps 401 to 407 may be performed prior to the vehicle leaving thefactory. Subsequent steps may be steps in a process of using thevehicle.

In 408, two driver monitoring assemblies are activated alternately.

During a process of using the vehicle, the two driver monitoringassemblies may be activated alternately to attempt to acquire a driver'sfacial posture information. Compared with simultaneously activating twodriver monitoring assemblies to acquire the driver's facial postureinformation, step 408 may reduce a load of the driver monitoringassemblies on an internal control device of the vehicle.

In 409, the facial posture information is continuously acquired by theany one driver monitoring assembly of the two driver monitoringassemblies, and the other driver monitoring assembly is deactivated, inresponse to the any one driver monitoring assembly of the two drivermonitoring assemblies detecting the facial posture information.

In the case that a driver monitoring assembly acquires facial postureinformation, it indicates that the driver is gazing at the drivermonitoring assembly, and then the facial posture information may becontinuously acquired by the driver monitoring assembly and the otherdriver monitoring assembly may be deactivated. In this way, only onedriver monitoring assembly is still running at the same time, which maynot cause a high load on the internal control device of the vehicle,thereby reducing the occupation of resources of the vehicle's internalcontrol device.

In 410, a visual field of the driver is determined based on the gazedirection of the driver and the eye position of the driver.

FIG. 9 is a schematic diagram of a visual field of the driver. As shownin FIG. 9 , the visual field is a quadrangular pyramid-shaped space, avertex d of the quadrangular pyramid is disposed at the eye position ofthe driver. In two groups of opposite apex angles of the quadrangularpyramid, one group of apex angles form a horizontal visual field angle“a” of the human eye, and the other group of apex angles form a verticalvisual field angle “b” of the human eye. A height direction of thequadrangular pyramid is parallel to the gaze direction of the driver.

In the case that an object is within the visual field, it may beconsidered that the driver is gazing at the object.

In addition, the above-mentioned quadrangular pyramid as the visualfield of the driver further includes a height parameter, which may bethe farthest distance the driver's eyes are able to see. The farthestdistance usually far exceeds internal dimensions of the vehicle, and maynot affect the method according to the embodiment of the presentdisclosure. Therefore, the height of the quadrangular pyramid may not beconsidered in the embodiment of the present disclosure.

It should be noted that the horizontal visual field angle and thevertical visual field angle of the driver's eyes may be preset universalvalues, or may be acquired by measuring the current driver, which is notlimited in the embodiment of the present disclosure. Optionally, thehorizontal visual field angle and the vertical visual field angle may beset to the same value, such as 60 degrees, so as to facilitatesubsequent calculation.

It should be noted that, in the embodiment of the present disclosure,the eye position of the driver may be a position between two eyes of thedriver, so as to simplify a calculation process and a calculationamount.

In 411, whether any one display assembly of the two display assembliesis within the visual field is determined based on the coordinates of thetwo display assemblies in the camera coordinate system.

In the case that the visual field of the driver is acquired, whether theany one display assembly of the two display assemblies is within thevisual field may be determined based on the coordinates of the twodisplay assemblies in the camera coordinate system.

Optionally, step 411 may include following two steps.

1) Whether edges of the any one display assembly of the two displayassemblies are within the visual field of the human eye is determined.

In the embodiment of the present disclosure, whether the human eye iscurrently gazing at a display assembly may be determined by determiningwhether all edges of the any one display assembly of the two displayassemblies are within the visual field of the human eye.

In the case that shapes of the display assemblies are different, theedges of the display assemblies may have different shapes. For example,edges of a rectangular display assembly may be rectangular, and whetherthe display assembly is within the visual field of the driver may bedetermined by determining whether four vertices of the display assemblyare within the visual field of the driver.

In the case that the driver is in the vehicle, the driver may have threestates of gazing at the display assembly on the left A-pillar, gazing atthe display assembly on the right A-pillar, and not gazing at the twodisplay assemblies. In this step, whether the driver is in a state ofgazing at a display assembly may be determined.

For example, FIG. 10 is a schematic diagram in the case that the displayassembly on the left A-pillar is within the visual field of the driver.As shown in FIG. 10 , it may be seen that all edges of the displayassembly 13 on the left A-pillar are completely within the visual fieldx of the driver.

2) In the case that all edges of the any one display assembly are withinthe visual field of the driver, it is determined that the any onedisplay assembly is within the visual field of the driver.

In the case that all edges of any one display assembly are within thevisual field of the driver, it may be determined that the any onedisplay assembly is within the visual field of the driver.

In 412, an external image of the vehicle captured by the imagingassembly is captured based on the eye position of the driver in responseto the any one display assembly being within the visual field, and thecaptured image is displayed by the any one display assembly.

In the case that the any one display assembly is within the visual fieldof the driver, the external image of the vehicle may be captured basedon the eye position of the driver (and a position of the displayassembly) to acquire a first sub-image in the driver's gaze direction inthe external image of the vehicle. In the case that the sub-image isdisplayed on the any one display assembly, a combined image may beformed by splicing the sub-image with an unblocked image around theA-pillar currently seen by the driver (that is, an image seen by thedriver through window glasses of the vehicle).

In summary, in the display method for the A-pillar-mounted displayassemblies of the vehicle according to embodiments of the presentdisclosure, by a conversion relationship between a coordinate system ofa target feature model and a camera coordinate system of a drivermonitoring assembly and a conversion relationship between the targetfeature model and a world coordinate system, a conversion relationshipbetween the camera coordinate system and the world coordinate system isacquired, the coordinates of the display assembly in the worldcoordinate system can be converted into the coordinates in the cameracoordinate system, such that whether a driver is gazing at the displayassembly can be simply determined. In this way, the problem in therelated art that it is difficult to determine, directly based on spatialpositions the driver's eyes and of the display assembly, whether thedriver is gazing at the display assembly is solved, and the effect ofreducing the difficulty of determining whether the driver is gazing atthe display assembly is achieved.

FIG. 11 is a structural block diagram of a display system for A-pillarsof a vehicle according to an embodiment of the present disclosure. Thedisplay system includes:

-   -   a posture acquiring module 1110, configured to acquire facial        posture information of a driver of the vehicle in a camera        coordinate system by any one driver monitoring assembly of the        at least one driver monitoring assembly, wherein the facial        posture information includes a gaze direction of the driver and        an eye position of the driver;    -   a visual field determining module 1120, configured to determine        a visual field of the driver based on the gaze direction of the        driver and the eye position of the driver;    -   a coordinate acquiring module 1130, configured to acquire        coordinates of the two display assemblies in a world coordinate        system;    -   a coordinate converting module 1140, configured to convert the        coordinates of the two display assemblies in the world        coordinate system into coordinates in the camera coordinate        system based on a first conversion relationship, wherein the        first conversion relationship is determined by a second        conversion relationship and a third conversion relationship, the        second conversion relationship is a conversion relationship        between the coordinates in the camera coordinate system of the        any one driver monitoring assembly and the coordinates in a        target feature model in the vehicle, and the third conversion        relationship is a conversion relationship between the        coordinates in the target feature model and the coordinates in        the world coordinate system;    -   a visual field determining module 1150, configured to determine,        based on the coordinates of the two display assemblies in the        camera coordinate system, whether any one display assembly of        the two display assemblies is within the visual field; and    -   an image displaying module 1160, configured to capture, based on        the eye position of the driver, an external image of the vehicle        captured by the imaging assembly in response to the any one        display assembly being within the visual field, and display the        captured image on the any one display assembly.

In summary, in the display system for A-pillars of a vehicle accordingto embodiments of the present disclosure, by a conversion relationshipbetween a coordinate system of a target feature model and a cameracoordinate system of a driver monitoring assembly and a conversionrelationship between the target feature model and a world coordinatesystem, a conversion relationship between the camera coordinate systemand the world coordinate system is acquired, the coordinates of thedisplay assembly in the world coordinate system can be converted intothe coordinates in the camera coordinate system, such that whether adriver is gazing at the display assembly is simply determined. In thisway, the problem in the related art that it is difficult to determine,directly based on spatial positions of the driver's eyes and the displayassembly, whether the driver is gazing at the display assembly issolved, and the effect of reducing the difficulty of determining whetherthe driver is gazing at the display assembly is achieved.

Further, an embodiment of the present disclosure further provides adisplay device for A-pillars of a vehicle. The display device includes aprocessor and a memory, the memory stores at least one instruction, atleast one program, a code set or an instruction set; wherein theprocessor, when loading and executing the at least one instruction, theat least one program, the code set or the instruction set, is caused toperform the display method for the A-pillar-mounted display assembliesof the vehicle according to any one of the above-mentioned embodiments.

An embodiment of the present disclosure further provides a storagemedium. The storage medium stores at least one instruction, at least oneprogram, a code set or an instruction set: wherein the at least oneinstruction, the at least one program, the code set or the instructionset, when loaded and executed by a processor, causes the processor toperform the display method for the A-pillar-mounted display assembliesof the vehicle according to any one of the above-mentioned embodiments.

In several embodiments according to the present disclosure, it should beunderstood that the disclosed device and method may be implemented inother ways. For example, the described device embodiments are merelyillustrative. For example, the units are defined merely in terms oflogical functionality, and may be actually practiced in other fashions.For example, a plurality of units or components may be combined or maybe integrated into another system, or some features may be ignored ornot implemented. In addition, the illustrated or discussed mutualcoupling, direct coupling or communication connection may be indirectcoupling or communication connection through some interfaces, devices orunits, and may be in electrical, mechanical or other forms.

The units described above as separate components may or may not bephysically separated, and the components illustrated as units may or maynot be physical units, that is, the units may be disposed in one place,or they may be distributed on a plurality of network units. Some or allof the units may be selected based on actual needs to achieve objectivesof solutions in embodiments of the present disclosure.

Those skilled in the art may understand that all or part of the steps inthe above embodiments may be implemented by hardware, or by a programinstructing related hardware. The program may be stored in acomputer-readable storage medium. The above-mentioned storage medium maybe a read-only memory, a magnetic disk or an optical disk, etc.

The described above are only optional embodiments of the presentdisclosure and are not intended to limit the present disclosure. Anymodification, equivalent replacement, improvement, and the like madewithin the spirit and principle of the present disclosure shall bewithin the protection scope of the present disclosure.

What is claimed is:
 1. A display method for A-pillar-mounted displayassemblies of a vehicle, wherein the method is applicable to thevehicle, the vehicle comprising an imaging assembly, at least one drivermonitoring assembly, and two display assemblies disposed respectively ontwo A-pillars at inside of the vehicle, and the method comprises:establishing a world coordinate system; acquiring a target image of atarget feature model in the vehicle by any one driver monitoringassembly of the at least one driver monitoring assembly; determining asecond conversion relationship between coordinates in the target featuremodel and coordinates in a camera coordinate system of the any onedriver monitoring assembly based on the target image of the targetfeature model; acquiring a third conversion relationship between thecoordinates in the target feature model and coordinates in the worldcoordinate system; determining a first conversion relationship based onthe third conversion relationship and the second conversionrelationship; acquiring facial posture information of a driver of thevehicle in the camera coordinate system by any one driver monitoringassembly of the at least one driver monitoring assembly, wherein thefacial posture information comprises a gaze direction of the driver andan eye position of the driver; determining a visual field of the driverbased on the gaze direction of the driver and the eye position of thedriver; acquiring coordinates of the two display assemblies in the worldcoordinate system; converting the coordinates of the two displayassemblies in the world coordinate system into coordinates in the cameracoordinate system based on the first conversion relationship;determining, based on the coordinates of the two display assemblies inthe camera coordinate system, whether any one display assembly of thetwo display assemblies is within the visual field; and capturing, basedon the eye position of the driver, an external image of the vehiclecaptured by the imaging assembly in response to the any one displayassembly being within the visual field, and displaying the capturedimage on the any one display assembly.
 2. The method according to claim1, wherein the target feature model comprises a standard facial featuremodel, the standard facial feature model comprising a plurality offeature points and three-dimensional coordinates of each of the featurepoints; acquiring the target image of the target feature model in thevehicle by the any one driver monitoring assembly comprises: acquiring atarget image of the standard facial feature model by the any one drivermonitoring assembly; and determining the second conversion relationshipbetween the coordinates in the target feature model and the coordinatesin the camera coordinate system based on the target image of the targetfeature model comprises: determining planar coordinates of the pluralityof feature points in the standard facial feature model based on thetarget image; acquiring an internal parameter matrix and a distortioncoefficient of the any one driver monitoring assembly by calibrating theany one driver monitoring assembly; and acquiring the second conversionrelationship by solving a perspective-n-point problem based on theinternal parameter matrix, the distortion coefficient, thethree-dimensional coordinates, and the planar coordinates.
 3. The methodaccording to claim 2, wherein acquiring the third conversionrelationship between the coordinates in the target feature model and thecoordinates in the world coordinate system comprises: acquiring arotation matrix and a translation vector between the standard facialfeature model and the world coordinate system; and determining the thirdconversion relationship based on the rotation matrix and the translationvector.
 4. The method according to claim 3, wherein prior to acquiringthe rotation matrix and the translation vector between the standardfacial feature model and the world coordinate system, the method furthercomprises: adjusting a coordinate system of the standard facial featuremodel such that one coordinate axis in the coordinate system of thestandard facial feature model is parallel to one coordinate axis in theworld coordinate system.
 5. The method according to claim 1, wherein thevehicle comprises two driver monitoring assemblies respectively disposedon the two A-pillars at the inside of the vehicle, the two drivermonitoring assemblies comprising the any one driver monitoring assemblyand the other driver monitoring assembly; and acquiring the facialposture information of the driver of the vehicle in the cameracoordinate system by the any one driver monitoring assembly of the atleast one driver monitoring assembly comprises: activating the twodriver monitoring assemblies alternately; and continuously acquiring thefacial posture information by the any one driver monitoring assembly ofthe two driver monitoring assemblies, and deactivating the other drivermonitoring assembly of the two driver monitoring assemblies other thanthe any one driver monitoring assembly, in response to the any onedriver monitoring assembly of the two driver monitoring assembliesdetecting the facial posture information.
 6. The method according toclaim 1, wherein the target feature model comprises a standard facialfeature model, the standard facial feature model comprising a pluralityof feature points and three-dimensional coordinates of each of thefeature points; and prior to acquiring the facial posture information ofthe driver of the vehicle in the camera coordinate system by the any onedriver monitoring assembly of the at least one driver monitoringassembly, the method further comprises: establishing the worldcoordinate system; acquiring a target image of the standard facialfeature model by the any one driver monitoring assembly; determiningplanar coordinates of the plurality of feature points in the standardfacial feature model based on the target image; acquiring an internalparameter matrix and a distortion coefficient of the any one drivermonitoring assembly by calibrating the any one driver monitoringassembly; acquiring the second conversion relationship by solving aperspective-n-point problem based on the internal parameter matrix, thedistortion coefficient, the three-dimensional coordinates, and theplanar coordinates; acquiring a rotation matrix and a translation vectorbetween the standard facial feature model and the world coordinatesystem; determining the third conversion relationship based on therotation matrix and the translation vector; and determining the firstconversion relationship based on the third conversion relationship andthe second conversion relationship.
 7. The method according to claim 1,wherein the imaging assembly comprises imaging sub-assemblies disposedat the outside of the two A-pillars, and a line between the imagingsub-assembly on any one A-pillar of the two A-pillars and the driver'seyes is intersected with the display assembly on the any one A-pillar.8. A display device for A-pillars of a vehicle, comprising a processorand a memory storing at least one instruction, at least one program, acode set or an instruction set therein; wherein the processor, whenloading and executing the at least one instruction, the at least oneprogram, the code set or the instruction set, is caused to perform adisplay method for A-pillar-mounted display assemblies of a vehicle, themethod comprising: establishing a world coordinate system; acquiring atarget image of a target feature model in the vehicle by any one drivermonitoring assembly of at least one driver monitoring assembly;determining a second conversion relationship between coordinates in thetarget feature model and coordinates in a camera coordinate system ofthe any one driver monitoring assembly based on the target image of thetarget feature model; acquiring a third conversion relationship betweenthe coordinates in the target feature model and coordinates in the worldcoordinate system; determining a first conversion relationship based onthe third conversion relationship and the second conversionrelationship; acquiring facial posture information of a driver of thevehicle in the camera coordinate system by any one driver monitoringassembly of the at least one driver monitoring assembly, wherein thefacial posture information comprises a gaze direction of the driver andan eye position of the driver; determining a visual field of the driverbased on the gaze direction of the driver and the eye position of thedriver; acquiring coordinates of the two display assemblies in the worldcoordinate system; converting the coordinates of the two displayassemblies in the world coordinate system into coordinates in the cameracoordinate system based on the first conversion relationship;determining, based on the coordinates of the two display assemblies inthe camera coordinate system, whether any one display assembly of thetwo display assemblies is within the visual field; and capturing, basedon the eye position of the driver, an external image of the vehiclecaptured by the imaging assembly in response to the any one displayassembly being within the visual field, and displaying the capturedimage on the any one display assembly.
 9. The display device accordingto claim 8, wherein the target feature model comprises a standard facialfeature model, the standard facial feature model comprising a pluralityof feature points and three-dimensional coordinates of each of thefeature points; acquiring the target image of the target feature modelin the vehicle by the any one driver monitoring assembly comprises:acquiring a target image of the standard facial feature model by the anyone driver monitoring assembly; and determining the second conversionrelationship between the coordinates in the target feature model and thecoordinates in the camera coordinate system based on the target image ofthe target feature model comprises: determining planar coordinates ofthe plurality of feature points in the standard facial feature modelbased on the target image; acquiring an internal parameter matrix and adistortion coefficient of the any one driver monitoring assembly bycalibrating the any one driver monitoring assembly; and acquiring thesecond conversion relationship by solving a perspective-n-point problembased on the internal parameter matrix, the distortion coefficient, thethree-dimensional coordinates, and the planar coordinates.
 10. Thedisplay device according to claim 9, wherein acquiring the thirdconversion relationship between the coordinates in the target featuremodel and the coordinates in the world coordinate system comprises:acquiring a rotation matrix and a translation vector between thestandard facial feature model and the world coordinate system; anddetermining the third conversion relationship based on the rotationmatrix and the translation vector.
 11. The display device according toclaim 10, wherein prior to acquiring the rotation matrix and thetranslation vector between the standard facial feature model and theworld coordinate system, the method further comprises: adjusting acoordinate system of the standard facial feature model such that onecoordinate axis in the coordinate system of the standard facial featuremodel is parallel to one coordinate axis in the world coordinate system.12. The display device according to claim 8, wherein the vehiclecomprises two driver monitoring assemblies respectively disposed on thetwo A-pillars at the inside of the vehicle, the two driver monitoringassemblies comprising the any one driver monitoring assembly and theother driver monitoring assembly; and acquiring the facial postureinformation of the driver of the vehicle in the camera coordinate systemby the any one driver monitoring assembly of the at least one drivermonitoring assembly comprises: activating the two driver monitoringassemblies alternately; and continuously acquiring the facial postureinformation by the any one driver monitoring assembly of the two drivermonitoring assemblies, and deactivating the other driver monitoringassembly of the two driver monitoring assemblies other than the any onedriver monitoring assembly, in response to the any one driver monitoringassembly of the two driver monitoring assemblies detecting the facialposture information.
 13. The display device according to claim 8,wherein the target feature model comprises a standard facial featuremodel, the standard facial feature model comprising a plurality offeature points and three-dimensional coordinates of each of the featurepoints; and prior to acquiring the facial posture information of thedriver of the vehicle in the camera coordinate system by the any onedriver monitoring assembly of the at least one driver monitoringassembly, the method further comprises: establishing the worldcoordinate system; acquiring a target image of the standard facialfeature model by the any one driver monitoring assembly; determiningplanar coordinates of the plurality of feature points in the standardfacial feature model based on the target image; acquiring an internalparameter matrix and a distortion coefficient of the any one drivermonitoring assembly by calibrating the any one driver monitoringassembly; acquiring the second conversion relationship by solving aperspective-n-point problem based on the internal parameter matrix, thedistortion coefficient, the three-dimensional coordinates, and theplanar coordinates; acquiring a rotation matrix and a translation vectorbetween the standard facial feature model and the world coordinatesystem; determining the third conversion relationship based on therotation matrix and the translation vector; and determining the firstconversion relationship based on the third conversion relationship andthe second conversion relationship.
 14. The display device according toclaim 8, wherein the imaging assembly comprises imaging sub-assembliesdisposed at the outside of the two A-pillars, and a line between theimaging sub-assembly on any one A-pillar of the two A-pillars and thedriver's eyes is intersected with the display assembly on the any oneA-pillar.
 15. A non-volatile computer storage medium storing at leastone instruction, at least one program, a code set or an instruction settherein; wherein the at least one instruction, the at least one program,the code set or the instruction set, when loaded and executed by aprocessor, causes the processor to perform a display method forA-pillar-mounted display assemblies of a vehicle, the method comprising:establishing a world coordinate system; acquiring a target image of atarget feature model in the vehicle by any one driver monitoringassembly of at least one driver monitoring assembly; determining asecond conversion relationship between coordinates in the target featuremodel and coordinates in a camera coordinate system of the any onedriver monitoring assembly based on the target image of the targetfeature model; acquiring a third conversion relationship between thecoordinates in the target feature model and coordinates in the worldcoordinate system; and determining a first conversion relationship basedon the third conversion relationship and the second conversionrelationship; acquiring facial posture information of a driver of thevehicle in the camera coordinate system by any one driver monitoringassembly of the at least one driver monitoring assembly, wherein thefacial posture information comprises a gaze direction of the driver andan eye position of the driver; determining a visual field of the driverbased on the gaze direction of the driver and the eye position of thedriver; acquiring coordinates of the two display assemblies in the worldcoordinate system; converting the coordinates of the two displayassemblies in the world coordinate system into coordinates in the cameracoordinate system based on the first conversion relationship;determining, based on the coordinates of the two display assemblies inthe camera coordinate system, whether any one display assembly of thetwo display assemblies is within the visual field; and capturing, basedon the eye position of the driver, an external image of the vehiclecaptured by the imaging assembly in response to the any one displayassembly being within the visual field, and displaying the capturedimage on the any one display assembly.
 16. The non-volatile computerstorage medium according to claim 15, wherein the target feature modelcomprises a standard facial feature model, the standard facial featuremodel comprising a plurality of feature points and three-dimensionalcoordinates of each of the feature points; acquiring the target image ofthe target feature model in the vehicle by the any one driver monitoringassembly comprises: acquiring a target image of the standard facialfeature model by the any one driver monitoring assembly; and determiningthe second conversion relationship between the coordinates in the targetfeature model and the coordinates in the camera coordinate system basedon the target image of the target feature model comprises: determiningplanar coordinates of the plurality of feature points in the standardfacial feature model based on the target image; acquiring an internalparameter matrix and a distortion coefficient of the any one drivermonitoring assembly by calibrating the any one driver monitoringassembly; and acquiring the second conversion relationship by solving aperspective-n-point problem based on the internal parameter matrix, thedistortion coefficient, the three-dimensional coordinates, and theplanar coordinates.
 17. The non-volatile computer storage mediumaccording to claim 16, wherein acquiring the third conversionrelationship between the coordinates in the target feature model and thecoordinates in the world coordinate system comprises: acquiring arotation matrix and a translation vector between the standard facialfeature model and the world coordinate system; and determining the thirdconversion relationship based on the rotation matrix and the translationvector.